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ABSTRACT 

The purpose of this study was to discover the 
efficacy of the Science, Mathematics, and Related Technologies 
Quality Measurement (SMARTQM) as a measure of total school quality 
and to determine the usefulness of this instrument in predicting 
school oriiiance through self-reported student achievement. SMARTQM 
is an integrated data collection and reporting system designed to 
determine how closely a school resembles the "ideal" school. It 
consists of 13 survey forms, site and classroom observation, and 
focus group discussions designed to obtain a complete picture of a 
school. Participants in the process include teachers, administrators, 
non-teaching staff, school-based councils, parents, students, 
previous students, and community members. A total of 17 schools 
participated in the pilot testing of the SMARTQM of which 15 were 
sufficiently complete for inclusion in the study. Schools were 
selected to fill slots in a sampling matrix. Schools were surveyed, 
visited by an evaluation team, and school community personnel 
participated in focus groups. The evaluation team independently 
scored the school on the SMARTQM characteristics. Appendices contain 
sample cover letter and surveys, site visit observation forms, and 
focus group questionnaire and procedures. (Author/MKR) 
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A SCHOOL QUALITY PLANNING AND EVALUATION TOOL 

Science, Mathematics, and Related Technologies Quality Measurement 

SMARTQM 



Dr. Brian E. Daly Manager of Innovation and Evaluation, Kentucky Science and 

Technology Council, P.O. Box 1049, Lexington, Kentucky 40588 



ABSTRACT 



The purpose of this study \A^as to discover the efficacy of the Science, Mathematics, and 
Related Technologies Quality Measurement (SMARTQM) as i.'' measure of total school 
quality and to determine the usefulness of this instrument in predicting school 
performance through self-reported student achievement. 



SMARTQM is an integrated data collection and reporting system designed to determine 
ho\A^ closely a school resembles the "ideal" school. It consists of thirteen survey forms, 
site and classroom observation, and focus group discussions designed to obtain a 
complete picture of a school. Participants in the process include teachers, administrators, 
non-teaching staff, school-based councils, parents, students, previous students, and 
community members. 
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A total of 17 schools participated in the pilot testing of the SMARTQM of \A^hich 15 \A^ere 
sufficiently complete for inclusion in the study. Schools \A^ere selected to fill slots in a 
sampling matrix. Schools \A^ere surveyed, visited by an evaluation team, and school 
community personnel participated in focus groups. The evaluation team independently 
scored the school on the SMARTQM characteristics. 



INTRODUCTION 

The U.S. Department of Commerce recently announced a pilot program in education 
based on the Malcolm Baldrige Award. This program is in concert with efforts by 
educators at all levels to improve the performance of schools and, subsequently, 
students. The goal of the Malcolm Baldrige Award is to improve the systemic planning 
and delivery of services of educational institutions across the country. 

Using Malcolm Baldrige software devised to evaluate the "quality" of American firms, a 
survey process to evaluate the quality of Kentucky schools was developed by the author 
while working for the Kentucky Council on Higher Education. The resulting software had 
problems inherent in materials developed for industry and not schools. In 1993, the 
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author and Nancy Sherman of the evaluation office of the Kentucky Science and 
Technology Council, Inc. began development of a new instrument v/hich would be more 
directly attuned to the realities and needs of education and would provide a unique 
evaluation tool for the state’s systemic initiative funded by the National Science 
Foundation. 

Partnership for Reform Initiatives in Science and Mathematics - PRISM 

In September, 1992, a cooperative agreement was signed between the National Science 
Foundation and the Kentucky Department of Education establishing the "Partnership for 
Reform Initiatives in Science and Mathematics (PRISM)" under the State Systemic 
Initiatives Program. This joint venture builds upon the changes taking place through the 
Kentucky Education Reform Act (KERA) to focus specific attention and resources on 
science and mathematics. PRlSM's goals are: 

* All Kentucky students will have the background in science and mathematics 
necessary to live and work in an increasingly complex society. 

* More Kentucky students, especially minorities, women, disabled and economically- 
disadvantaged persons, will choose to pursue science and mathematics-related 
careers. 

* Kentucky citizens and leaders will recognize the importance of quality science and 
mathematics education to the state's economic competitiveness. 

PRISM is a partnership effort among a large number of individuals and groups 
across Kentucky, representing both public and private sectors. The chief partners 
in this project are the Kentucky Science and Technology Council, Inc., the 
Kentucky Department of Education, and the state universities of Kentucky. 
Through coordinated activities, the Partnership for Reform Initiatives in Science 
and Mathematics is facilitating changes in the ways that science and mathematics 
are taught and learned in Kentucky schools. 

SMARTQM 

SMARTQM is an integrated data collection and reporting system designed to 
determine how closely a school resembles an ’Ideal" school. The underlying 
assumption is that excellence in science, mathematics, and related technologies 
can be recognized and that schools can be compared against a vision of the ideal. 
Although no school will have all the attributes of an "ideal" school, many will have 
components which exemplify excellence. Essentially, the SMARTQM is a 
strategic planning process using the format of the Malcolm Baldrige National 
Quality Award, the structure of Total Quality Management, and the results of 
exemplary schools research. 
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SMARTQM consists of thirteen survey forms, schooi and classroom observation, 
and focus group discussions designed to obtain a picture of the entire schooi. 

Participants in the process include teachers, administrators, non-teaching staff, 
school-based council members, parents, students, previous students, and, 
optionally, community members. Specific attention is given to the performance of 
the school in science and mathematics. Classroom observation and four of the 
survey forms are specifically targeted toward science and mathematics. In 
addition, a section of the Previous Student Survey is tied directly to the science 
and mathematics outcomes identified in Transformations: Kentucky’s Curriculum 
Framework. Finally, nine of the survey forms allow the school to design and 
include statements which target specific concerns or areas of interest. 

After the site visit, participating schools receive a written report which provides 
scores on fourteen criteria; the seven Malcolm Baldrige Award criteria and seven 
SMARTQM Exemplary School Processes. In addition, specific questions from the 
surveys are highlighted to give the school some direct feedback on what students, 
faculty, parents, and graduates think about the school. The data from this report 
are used to generate practical suggestions on what can be done to improve the 
school's operation and overall effectiveness. Specific categories of the SMARTQM 
Report are defined below. 



A. Malcolm Baldrige Award Criteria 

Leadership (95 points) -The school provides the necessary support and focus to 
accomplish the goals of a quality school. 

Information and Analysis (75 points) - The school has clearly defined methods to 
systematically look at how its systems are functioning. 

Strategic Quality Planning (60 points) - The school has a system of planning which 
involves the appropriate groups, focuses on short and long-term objectives, and 
stresses continual improvement. 

Human Resource Utilization (150 points) - The school hires, develops, involves, 
empowers and recognizes quality personnel. 

Quality Assurance of Students and Services (140 points) - The school has a 
definition of quality student outcomes and quality services for students, continually 
strives for consumer satisfaction, and has a system for evaluating its students and 
services. 

Quality Results (180 points) - The school maintains measures of quality which 
compare it with the very best schools. 
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Customer Satisfaction (30G points) - The school has a defined process to study 
utilize, and evaluate feedback from its different constituencies. 



B. SMARTQM Exemplary School Processes 

Curriculum and Standards - An operational plan for instruction reflecting value 
judgments that enumerate \A^hat students need to know; how students are to 
achieve curricular goals; what teachers do to help students develop; and the 
context in which learning and teaching occur. 

Teaching - Actions by teachers which enable students to meet and exceed 
identified curricular goals. 

Relationships - Personal interactions which provide support, encourage 
achievement, enrich the curriculum, and strengthen the teaching/learning process. 

Self and Professional Development - Teachers and staff actively seek, and the 
school provides, access to educational activities and opportunities which facilitate 
and increase achievement of curricular goals. 

Use of Resources and Personnel - The active and imaginative use of persons, 
places, and materials which facilitate and increase achievement of curricular goals. 

Empowerment - An active, personal commitment to assumin.g and sharing 
responsibility. 

Equity- An active recognition and valuing of individual uniqueness, which accepts 
no artificial barriers to realizing potential. 



C. SMARTQM - Other Highlights 

Student Perceptions - Attitudes toward school, learning, teachers, safety, and 
personal abilities. 

Faculty/Staff Perceptions- Attitudes toward the school, learning, students, parents, 
commitment to quality, professional development, and the work environment. 

Graduate (Previous Student) Perceptions - Attitudes toward preparation for the 
next educational level; self-reported performance in writing, history, mathematics, 
and science; and ratings of the former school and teachers. 

The information is graphically presented in a series of bar charts. Comparisons 
between how the different groups perceive the school ara portrayed. 
Recommendations by the PRISM team are included. 
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SMARTQM - Sample Report Format 
TABLE 1 - Quality Indicators 



Indicator 


Teachers 


Students 


Graduates 


Parents 


Overall 


Leadership 


3.0 


3.3 


3.0 


2.9 


3.1 


Information & Analysis 


3.2 


3.1 


3.0 


2.1 


3.0 


Quality Planning 


3.1 


2.3 




1.8 


2.9 


Human Resources 


2.8 


3.3 






2.9 


Quality Assurance 


2.8 




2.9 


2.3 


2.8 


Quality Results 


2.8 


2.8 


2.8 


3.2 


2.9 


Client Satisfaction 


2.8 


2.9 


2.9 


3.4 


2.9 



The Malcolm Baldrige Award Criteria give a weight to the different indicators with a 
possible perfect score of 1,000. This school scored 728 points. The Science & 
Mathematics component scored 670 points. 

Parents believe that: 



A similar table is presented for SMARTQM Process Scores. 



■ innovative education can improve their child's education (84 percent);- 

■ their child is happy in school (89 percent); 

■ their child enjoys learning (91 percent); 

Some general recommendations based on this analysis include the following: 



Data summaries and bar graphs are shown for teachers, students, and graduates. 



1. Parents need to be recruited and involved in school planning. This could be done by 
running small invitational focus groups at convenient times. The purpose of these groups 
would be to provide parents with information about how reform is working in their school 
and to gather their ideas about how the school can do a better job. This process will 
open communication channels and encourage greater understanding. — ETCETERA — 



ASSESSMENT INSTRUMENTS 
SURVEY FORMS 



The first component of SMARTQM consists of thirteen survey forms and involves 
participation by teachers, administrators, school-based councils, non-teaching staff, 
students, parents, previous students, and community members. Distribution of the forms 
is as follows; 



SURVEY NUMBER & TITLE 


TARGET AUDIENCE 


1 - Teacher Self-Assessment 


All Teachers 


2 - School Administration Survey -Teachers 


All Teachers 


3 - Science Component Survey 


Science Teachers 


4 - Mathematics Component Survey 


Math Teachers 


5 - School Environment Survey 


Administrators 

Teachers 

Staff 

Volunteers 


6 - Administration Survey 


Administrators 

School-Based Council Members 
Teachers in Administrative Rotes 


7 - School Administration Survey - Staff 


Non-Teaching & Support Staff 


8 - Student Survey 


Students - Grades 3 and Higher 


9 - Student Science Survey 


Science Students 


10 - Student Math Survey 


Math Students 


11 - Previous Student Survey 


Previous Spring Graduates 


12 - Parent Survey 


Parents 


13 - Community Survey 


Business & Community Leaders 



SMARTQM - SURVEYS 

Introduction 

Each of the surveys consists of statements which require an agreement rating from low 
to high. For most questions, respondents are directed to choose one best response. On 
several forms, additional demographic or open-ended questions are asked. Open-ended 
questions are designed to provide the school with some immediate feedback and, usually, 
the data from these questions will not be included in the SMARTQM Report. 

Nine of the surveys include space for additional questions to address specific concerns 
or interests of the school. Response patterns to these questions will be included in the 
SMARTQM Report if the school provides copies of the questions to the PRISM team. If 
the school decides not to use this portion of the survey, this section of the answer sheet 
should remain blank. Distribution of the surveys differs for each questionnaire. A 
sample cover letter, directions for completing a survey, and sample surveys are found in 
Appendix A. 



SMARTQM - SITE AND CLASSROOM OBSERVATIONS 

The second component of SMARTQM consists of site and classroom observations. 
Scheduled visits to science and mathematics classes provide an opportunity for direct 
observation of school characteristics as well as teacher and student behaviors. Library 
and computer facilities and materials are included in the site observation. A copy of the 
classroom and the facility observation forms are attached as Appendix B. 

SMARTQM - FOCUS GROUPS 

The third component of SMARTQM consists of focus group data from faculty and staff 
members, students, and, optionally, parents and community members. Members of the 
focus groups discuss and prioritize the characteristics of the ideal school, and identify the 
characteristics of their own school. 

Focus group data are used to verify and supplement data gathered through the survey 
process. Normally, the PRISM team will conduct student and teacher focus groups 
during their visit to the school. When possible, parent and/or community focus groups 
can be conducted by the PRISM team. Due to scheduling difficulties, these parent and 
community focus groups may be more effectively conducted by school personnel using 
the SMARTQM structure. Results of the discussions could then be used by the school 
for local planning activities. 
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RESEARCH DESIGN 



Sample: The sample for this study consists of 13 public and 2 private schools 
geographically distributed across the state of Kentucky representing different socio- 
economic strata, population densities, ethnic compositions, and involvement in the 
statewide systemic initiative funded by the National Science Foundation. Private schools 
were selected because no mandates exist for them to initiate education reform. 

The various surveys which make up the SMARTQM are directed to specific audiences 
at each school. The n's for the various subgroups is as follows: 



Teachers 322 

Science Teachers 144 

Mathematics Teachers 157 

Students 1183 

Non-Education Support Staff 115 

Administrators & Council Members 94 

Graduates & Previous Students 572 

Parents 817 

Community Members 112 



In addition, an approximate total of 300 students and 300 teachers participated in focus 
groups. 

Procedures: Surveys were compiled for each school to form a matrix of 
characteristics. Using a judgement analysis process, questions were categorized into 
the 14 criteria (7 Malcolm Baldrige and 7 SMARTQM Indicators) and scores were 
calculated based on these groupings. Seven groupings were partially derived from 
previously defined Malcolm Baldrige question groups. 

An independent rating of each school on the 14 criteria was done by each of the team 
members. These independent ratings were then combined to arrive at a consensus 
score for each school. 

The consensus score and the SMARTQM generated mean were then compared using 
Pearson Product Moment Correlation and a simple T-Test. The consensus score was 
also compared with the graduate generated self-assessment of performance at the 
next level institution. 

Each school was given an overall score on the Malcolm Baldrige criteria based on the 
different point values for each criterion. School scores ranged from 622 to 812 on a 
1,000 point scale. These scores were then compared with student self-reported 
success at the next level of education. A Pearson product moment correlation was 
run to determine the strength of relationships. 



RESULTS 



The 14 criteria of the SMARTQM showed a weak relationship with the 14 consensus 
scores. Pearson correlations ranged from a low of zero with a high of .84 to a low of 
.07. The mean correlation for the set was .40. 

The Cronbach's Alpha demonstrated survey item reliability from a low of .90 to a high 
of .96. The mean alpha for the 13 surveys was .93 while the mean inter-item 
correlation was .30. 

A Pearson product moment correlation between students self-reported scores and 
their subsequent performance at the next higher level school showed that self reported 
student performance at the next level and school Malcolm Baldrige overall scores are 
not directly related. 



Discussion: The results of the initial analysis point out three important factors: 

1 . The inter-rater reliability was poor. The team members did not have consistent 
ratings on the 14 evaluation items either internally nor externally. Their consensus 
scores were significantly different from all the SMARTQM process indicatoi scores and 
similar to only three of the Malcolm Baldrige scores. The results leave a significant 
question about the validity of the data. 

2. The reliability of the surveys was high. Alpha reliabilities were above .90 with 
reasonably low inter-item correlations. 

3. The use of this instrument to predict the performance of students is not supported 
by the data. The instrument is designed to measure school factors and self-reported 
student scores are too individualized. This same analysis against independent 
statewide scores may also reflect a lack of concurrence. The type of changes 
measured by the SMARTQM are based on long-term goals. Institutions which apply 
quality leadership may demonstrate student gains in future years but the relationship 
is probably a weak one. 

Ramifications: The instrument in its current configuration is limited to specific 
applications. The validity question must be determined before extensive use of the 
instrument as a meucure of quality implementation is possible. Although valuable data 
were made available through the extensive survey process, the 14 indicator scores 
are suspect. The focus group data provided planners with important feedback 
concerning needs. 

One major challenge is to improve the inter-rater reliability. Training of leam members 
in the 14 criteria through better definitions and practice sessions needs to be 



8 



n 



conducted. Although the current process may offer a positive indicator of a school's 
strengths and weaknesses, until a reliable cross-measure is developed, the validity of 
the instrument remains in question. 

Interestingly factor analysis seems to support at least ten of the fourteen indicators. 
Further analyses in this area will strengthen the internal validity of the instrument. 
Better rating training will provide a better check on external validity. 

Conclusions: The SMARTQM provides a unique view of schools. It offers survey 
data which individually can help a school plan effective for education reform. Its true 
strength, however, will emerge when it can be used to report derived scores which 
point to specific areas needing attention. The current version needs to be better 
refined before any broader use is undertaken. 




9 




APPENDIX A 



SAMPLE COVER LETTER & SURVEYS 

To: All Teachers 

From: 

Date: 

Re: SMARTQM Teacher Self-Assessment Survey 



Our school is participating in a quality analysis called the Science, Mathematics and 
Related Technologies Quality Measurement (SMARTQM) and 1 need your help. 
Please complete the attached survey. You are asked to determine, on a scale from 
\o\N to high, how closely each statement describes your attitudes and/or behaviors. 
Please use a pencil to fill the center of the appropriate circle. 

The results of this assessment will provide valuable information about our school. 
Your completed survey is confidential and only a summary report will be provided. 
Please be candid in your responses. Thanks for your help. 

Please return your completed form to the SMARTQM collection point in the school 
office. 



There are spaces on the form for several additional items of special interest to the 
school. Statements for items 35-38 can be found below. Please use the same low to 
high scale to complete this part of the survey. 



35. 



36. 

37. 



38. 




THANK YOU 

10 
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PARTNERSHIP for REFORM INITIATIVES in SCIENCE & MATHEMATICS (PRISM) O " 
SMARTQM Teacher Self-Assessment Survey " 



The following are statements about teaching at your school. Decide how strongly you agree with 
each, select one response, and fill the center of the appropriate circle/:P/ease use a pencil. 



SCHOOL: 



AGREEMENT SCALE: Low to High 



LOW- 



HIGH 



1 . i have a clearly defined focus for each instructional session. 

2. S reflect the belief that all students can learn and achieve success In my classes. 

3. 1 exhibit enthusiasm for the subject. 

4. 1 build Instruction on what students learned earlier. 

5. 1 move Instruction from concrete examples to abstract concepts. 



6. I provide supervised practice of Important skills. 

7. I regularly monitor student learning in class. 

8. i establish a pattern of success and achievement for each student. 

9. I encourage students to explain the process used to arrive at an answer. 

10. 1 regularly provide specific feedback to students In class. 



11.1 use a variety of assessment measures and techniques. 

12. 1 use activities which require solving non-routine problems from everyday life. 

13. 1 require students to express Ideas through written assignments. 

14. ! group students Into heterogeneous study or activity teams. 

15. 1 use models, simulations, and games regularly to teach concepts. 



1 6. i Insist that students collect and use evidence. 

17. 1 feel comfortable using a computer. 

18. i evaluate, select, and Integrate computers and technology into the subject area. 

19. 1 Individualize Instruction to match student learning styles and abilities. 

20. 1 have good rapport with the other teachers In my school. 



21. My colleagues and I meet to discuss how to handle problems with students. 

22. My colleagues and 1 share Ideas on different effective teaching approaches. 

23. My colleagues and 1 cooperatively organize teaching to be mutually supportive 

24. 1 upgrade my content knowledge through reading professional literature. 

25. 1 am an active member of a content area professional association. ; 



26. 1 upgrade my teaching abilities through professional development activities. 
27. i am an active member of an education professional association. 

28. 1 use or serve as a peer coach to assist teachers. 

29. 1 take an active role In curriculum development. 

30. 1 observe the classes of my colleagues. 



31. My colleagues and I conduct research on teaching and teaming. 

32. 1 provide Ideas to my superiors on how to Improve my school. 

33. 1 feel my school has a true commitment to Improving Its quality. 

34. AH things considered, students receive the best possible education at this school. 



35 

36 

37 



J 



39 . 



Specialty Area(s) u tsngusgs Arts SodatStudlss ^ a o tdisim 



Others 



ERiC) 

frwm Number 76610*B* 12 



9090 
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PARTNERSHIP for REFORM INITIATIVES in SCIENCE & MATHEMATICS (PRISM) O “ 
SMARTQM Student Science Survey ■■ 



The following are statements about science at your school. Decide how strongly you agree 
with each item, select one response, and fill the center of the appropriate circle. There 
are no right or wrong answers. Please use a pencil. 



SCHOOL: 



AGREEMENT SCALE: 



Low to High 

33. RACE 



LOW- 



1. My teachers tell me science is important. 

2. My parents tell me that science is important. 

3. My teachers tell me i can do well In science. 

4. My parents tell me 1 can do well in science. 

5. tt Is Important for me to know how to use science. 



6. 1 would like to spend more time on science. 

7. Science is often tun. 

8. My teacher enjoys science. 

9. 1 know 1 can do well In science. 

10. 1 could work successfully on harder science problems. 



11. 1 enjoy solving problems using science. 

12. There Is more than one way to solve many science problems. 

13. 1 would enjoy being in a science team competition. 

14. 1 like to play science games. 

15. 1 use a calculator In science class. 



16. 1 use a computer In science class. 

17. 1 enjoy using a calculator or computer for science. 

18. My science skills help me learn other subjects more easily. 

19. There Is a connection between science and other subjects. 

20. We do science experiments In our class. 



21. My parents ask about my science class. 

22. My teacher uses many different ways to teach science. 

23. My teacher asks me to explain how 1 solve science problems. 

24. My teacher uses **real life" examples to teach science. 

25. My teacher uses models and displays to explain science. 



:) 



26. My teacher helps me know my strengths and weaknesses in science. 

27. Science Is one of my favorite subjects. 



SPECIAL QUESTIONS 



28. 

29. 

30. 

31. 

32. 



What suggestions do you have to improve the teaching of science? 



^5 



HIGH o 

Caucasian 
(WhHe) 

Other Race 



34. GENDER 



Female 

Male 



35. CURRENT 
GRADE 



c 


y 9 


p ^ $ 


3 10 


5 V 


) 11 


6 C 




7 C 


) 13 


CD 8 C 


J) 14 



OVERALL RATING 
36. Scltncs T»ach#r 




37. Previous 
Scianct TMchart 




38. Othar Currant 

Taachara 

HIGH 



LOW 
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PARTNERSHIP for REFORM INITIATIVES in SCIENCE & MATHEMATICS (PRISM) O 
SMARTQM Previous Student Survey - Middle School 



The following are statements about your former school. Decide how strongly you agree with 
each Item, select one response, and fill the center of the appropriate circle. There are 
no right or wrong answers. Please use a pencil. 



PREVIOUS SCHOOL: 



AGREEMENT SCALE: Low to High 



LOW- 



HIGH 



1. I teel my tormer school prepared me well tor my current school. 

2. I would recommend my former school to a younger friend. 

3. I like my former school. 

4. I like my current school. 

5. My current school is more difficult. 




6. I have good language arts skills (reading, writing, and grammar). 

7. I am currently doing well (A or B) in my English class. 

8. I am well-prepared in math. 

9. I am currently doing well (A or B) in math. 

10. 1 am well-prepared in history and social studies. 



1 1 . 1 am doing well (A or B) in history/social studies. 

12. 1 am well-prepared in science. 

13. 1 am currently doing well (A or B) In science. 

14.1 find it easy to speak in front of the class. 

15. I know how to conduct an experiment. 



1 3. 1 find it easy to work in a small group on school work. 

17. Teachers at my former school told me I could do well in my classes. 

18. There was an adult at my former school with whom I could talk. 

19. 1 learned to use computers at my former school. 

20. Teachers at my former school used many ways to "test* my knowledge 



21 . 1 liked using the library at my former school. 

22. 1 think my former school treated all students fairly. 




How well do you think you would do In each of the following tasks? 

23. Develop a chart to show the things you like and dislike In a book. 

24. Listen to a speaker and pick out weaknesses in his/her arguments. 

25. Invent a system to feed and water a pet while a family is away on vacation. 

26. Design a survey, collect, analyze, and graph data; report the results. 

27. Use blocks to demonstrate fractions and decimals. 



LOW 



HIGH 



28. Return a defective tape recorder to a store. 

29. View a scene in a movie and describe appropriate and inappropriate behaviors. 

30. Tell which persons are most scared in a dentist's waiting room. 

31 . Observe people at a local store and record their non-verbal behavior. 

32. Design a print advertisement for your favorite song. 



33. Design and test a container in which to ship a fragile glass statue. 

34. Interview persons who were affected by unfair labor practices & report results. 

35. Invent a video game which shows how attitudes have changed since 1900. 

36. Communicate by computer with a person in another school. 




What one thing would you change about your former school? 



37. RACE 



C) African Am. 
(BLACK) 

C ) Caucasian 
(Whlta) 



o 



Other Race 



38. GENDER 




39. Number of Years 
at Former School 




OVERALL 

RATINGS 



40. Former School 




41. Former Teachers 

High 



Low 



42. Fortner Fadllfy 





ERIC 



Form Numliirr 750 10- 5- 12 
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APPENDIX B 



SITE VISIT OBSERVATION FORMS 



KENTUCKY PRISM - Teacher Observation Checklist 






Teacher: 



Class: 



Grade Level: 



INDICATE WHICH OF THE FOLLOWING ITEMS YOU OBSERVED 



Physical Environment 
\ 1 . Seating arranged in rows. 

. Seating arranged in ciustets. 

1 3. Seating in semi-circies or other anangement. 

1 4. Adequate & appropriate space for activities. 

1 5. Adequate books and materials availabiei 
. Student work displayed in room. 

1 7. Different natural or artistic modeb displayed. 

1 8 . Displays show muttl-culturai modem usage. 

1 9. Materials display ethnic/gender diversity. 

1 10. Adequate, secure storage for materials. 

111. Library has sufficient support materials. 

1 12. Appropriate computing resources available. 



Q27. 




Teacher Behaviors 
. Clearly defined focus for the session. 

. Reflects the belief that all students can ieam 
and achieve success in the subject. 

. Encourages student to explain the process 
used to arrive at an answer. 

. Builds instruction on earlier learning. 

. Moves Instruction from abstract to concrete. 
. Provides supervised practice of skills. 

. Integrates computers and technology into 
the subject area. 

. Gives individual attention to students. 

. Monitors student learning in class. 

. individualizes Instruction to fit student(s). 

. Exhibits enthusiasm for the subject. 

. Provides specific feedback to students. 

. Uses a variety of assessment techniques. 

. Uses activities which require solving non- 
routine problems from everyday life. 

Requires students to express ideas through 
writing assignments. 

Groups students into small learning teams. 
Uses games/simulations to teach concepts. 

. Has students collect and use evidence. 

. Involves all race and gender students In 
class discussions and activities. 

Exhibits good safety practices. 



Student Behaviors 
)33. Building and/or discussing models. 
)34. Measuring or estimating. 

)35. Working with physical models. 

)36. Gathering and interpreting data. 
)37. Making or reading graphs. 

38. Figuring out learning games. 

)39. Debating and discussing ideas, 
to. Explaining or showing others. 

41 . Asking questions 

42. Drawing diagrams or graphs. 

. Writing about results. 

. Using calculators. 

, Using computers. 

46. Working collaborativeiy In a team. 

47. Exhibiting enthusiasm for subject. 

, Relating the topic to the real world. 

49. Accepting ethnic/gender diversity, 
sa Focusing on learning tasics. 



OBSERVER SUGGESTIONS & COMMENTS 



Strengths: 



Weaknesses: 



1 10 
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KENTUCKY PRISM - Library & Technology Checklist 






Schooi: 



ENVIRONMENT 



Inviting appearance for patrons 
Use of book displays 
I 3. Area for individual study 
\ 4. Area for small group study 
Accessible location 
Sufficient space for two classes 
Adequate physical access to coliection 
' 8. Adequate resource search capability 
I 9. Furniture is appropriate 

10. Access to external databases 

1 1 . Automated reference capability 

12. Periodicals available for patrons 

13. Accessible for physically challenged 
1 14. Interlibraiy loan capability 



SERVICES 

I . Reference help availabfe 

I 2. Facility open throughout school day 
I 3. Facility open for extended school hours 
f 4. Professional staff available 

Clerical/technical support available 
I 6. Library orientation for students 
I 7. Library orientation for teachers 
I 8. Library events regjiariy published 
I 9. Scheduling service for fadities 
I 10. Provides equipment setup support 

II. Accessible catalog of AV materials 

1 12. Maintain equipment status 

1 13. Maintain library usage data 

1 14. Maintain media usage data 



PLANNING 

, 1 . Written goals arrd objectives 
i 2. Maintain site budget for library 
Plan for collection development 
\ 4. Plan for curriculim integration 
I 5. Library media acJvIsory committee 
I 6. Collaboration with teachers 
^ 7. Student input sought 
) 8. Parent input sought 
I 9. Cooperative planning with other libraries 



er|g 



Date: 

INSTRUCTIONAL SUPPORT 

I . Accurate and current resources 
Science Publications 
Math Publications 

Provide up-to-date technologies 
)3. Instructional production capability 
)A. Integration of information skills with curriculum 
6. Library resources In performance assessment 
6. Effective communication with teachers 
Assist professional development of teachers 
8. Provide current journals about teaching 
Provide current math periodicals for teachers 
1 0. Provide current science periodicals for teachers 

II . Teacher involvement in collection selection 

12. Develop teacher information access proficiency 

13. A positive learning environment for students 

14. Fast access to current information 

15. Motivational activities to promote reading 

16. Technology for student information access 

17. Technology for production of student materials 



SCHOOL COMPUTER RESOURCES 

)1 . School catalog of computer software 
)2. 1 to 6 student / computer ratio 
Updated student computer lab 
Updated computers in science classrooms 
Updated computers in math classiooms 
Updated computers in other classrooms 
)7. Modem capability in computer lab 
Modem capability in classrooms 
Computer training for faculty 
)10. Software choice assistance 
)1 1 . Technology planning team active 
)12. Published technology plan 
)13. Budget specifically for computers 
)14. Computers used by M&S teachers 
)15. Students have individual computer time 



COMMENTS 
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APPENDIX C 



FOCUS GROUP QUESTIONNAIRE & PROCEDURES 

FOCUS GROUP AGENDA 



introduction 

A. Divide the group into 2 to 4 teams (depending on the number of participants with 3-5 
in a group) 



B. Give each team a list of word cards (each word on a separate paper) for the four 
different components; Physical Facility; Curriculum: Teacher; and Student. 



C. Provide the following verbal instructions; 

1. The word cards I gave you list attributes of an ideal (school, curriculum, teacher, 
student). Think of other words which you think would describe this ideal and write 
them on the blank cards. 

2. Which 5 of these attributes are most important? Which 5 are least important? 

3. Which 5 of these attributes are best at this school? Which 5 are worst? 



D. Have each team rotate through the other team's cards and make one change in each 
stack of cards if they feel it is needed. 

E. Enter the data on the sheets, give 5 points for "high" words, 3 points for the "middle" 
words and 1 point for the "low" words. Subtract the actual from the ideal. (If there is time 
the focus group participants can help with this.) 



F. Take the results from each different focus group and average the differences. Items 
of concern are those items with the greatest differences. 
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SCHOOL: 

STUDENT DESCRIPTORS 



IDEAL 


DESCRIPTOR 


ACTUAL 




NICE - FRIENDLY 






SMART 






CREATIVE & INTERESTING 






FAIR 






RESPECTFUL 






PATIENT 






ORGANIZED 






COOPERATIVE 






HONEST 






CONFIDENT 






SENSE OF HUMOR 






MOTIVATED 






ADAPTIVE - WILUNG TO TRY NEW 
THINGS 
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